Random Projection Features and Generalized Additive Models

We propose to learn generalized additive models for classification which represents the classifier using a sum of piecewise linear functions and show that a recently proposed fast linear SVM training method (Pegasos) can be adapted to train such models with the same convergence rates. To be able to learn functions on combination of dimensions, we explore the use of random projection features which learns a classifier on data projected using an arbitrary matrix. In our experiments we find that : (i) The piecewise linear consistently outperforms linear on various datasets (ii) Random projection features perform even better and are close to the best results (using RBF kernels) while potentially being much faster to train on large datasets.